Hybrid deployments #563

radez · 2024-10-03T15:42:07Z

Baremetal control plane and virtual workers.
I think this will also do baremetal control plane and workers and virtual workers.

radez · 2024-10-03T15:42:38Z

Working on testing this. I think it "works" but I wanted some feedback of what I'm missing or not implementing correctly.

akrzos · 2024-10-04T15:51:10Z

ansible/roles/boot-iso/tasks/libvirt.yml

+- name: Libvirt - Pause for power down
+  pause:
+    seconds: 1
+  when: not redfish_forceoff.failed


Might be able to use a "check for powered down" type of task here instead of a sleep.

I was able to remove this entirely on my tested deployments (3, 27, and 54 VMs)

radez · 2024-10-08T13:27:51Z

re: issue #536

openshift-ci · 2024-11-22T17:17:10Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: radez

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [radez]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

akrzos · 2025-01-02T14:18:44Z

ansible/mno-deploy.yml

+  - role: boot-iso
+    vars:
+      inventory_group: hv_vm
+      index: "{{ hybrid_worker_count }}"


In the context of someone running an ACM scale test where all of the hv_vm entries are actually say SNOs, does this task dump thousands of lines of skipped tasks or does it just skip the role? If it dumps thousands of lines, I think we should revisit how this is performed perhaps using a different inventory group.

You're right it does dump thousands of lines. I had looked at adding a loop_var in this at one point to make the output more meaningful

I'm wondering if we could instead of including another boot-iso role here over hv_vm workers, maybe we just copy the desired number of hv_vm we want to use under workers instead. I will think of a more automated way to accomplish this as well. WDYT?

- Specify Podman as the deploy type for the bastion AI container example podman configmap: https://github.com/openshift/assisted-service/blob/master/deploy/podman/configmap.yml - no need to patch the cluster network settings after boot All the same settings are defined at cluster creation

akrzos

I was able to start building some clusters where my controlplane nodes were bare metal and my worker nodes were VMs however I have some feedback.

akrzos · 2025-01-17T14:11:23Z

ansible/roles/boot-iso/tasks/libvirt.yml

+
+- name: Libvirt - Check for Virtual Media
+  uri:
+    url: "http://{{ hostvars[item]['ansible_host'] }}:9000/redfish/v1/Managers/{{ hostvars[item]['domain_uuid'] }}/VirtualMedia/Cd"


I had to change Managers to Systems because Managers didn't work 🤣

akrzos · 2025-01-17T14:33:21Z

ansible/roles/boot-iso/tasks/libvirt.yml

+
+- name: Libvirt - Eject any CD Virtual Media
+  uri:
+    url: "http://{{ hostvars[item]['ansible_host'] }}:9000/redfish/v1/Managers/{{ hostvars[item]['domain_uuid'] }}/VirtualMedia/Cd/Actions/VirtualMedia.EjectMedia"


Same here with Managers

akrzos · 2025-01-17T14:33:28Z

ansible/roles/boot-iso/tasks/libvirt.yml

+
+- name: Libvirt - Insert virtual media
+  uri:
+    url: "http://{{ hostvars[item]['ansible_host'] }}:9000/redfish/v1/Managers/{{ hostvars[item]['domain_uuid'] }}/VirtualMedia/Cd/Actions/VirtualMedia.InsertMedia"


Same here with Managers

akrzos · 2025-01-17T14:34:22Z

ansible/roles/boot-iso/tasks/libvirt.yml

+- name: Libvirt - Pause for power down
+  pause:
+    seconds: 1
+  when: not redfish_forceoff.failed


I was able to remove this entirely on my tested deployments (3, 27, and 54 VMs)

akrzos · 2025-01-17T14:35:33Z

ansible/roles/create-ai-cluster/tasks/main.yml

+        "additional_ntp_source": "{{ bastion_controlplane_ip if use_bastion_registry else labs[lab]['ntp_server'] }}",
+        "api_vips": [{"ip": "{{ controlplane_network_api }}"}],
+        "ingress_vips": [{"ip": "{{ controlplane_network_ingress }}"}],
+        "network_type": "{{ networktype }}"


I believe we need to revert this.

I found the root of the original issue that was causing my deployment with this to fail PR. We can leave this as is.

akrzos · 2025-01-17T14:41:00Z

ansible/roles/wait-hosts-discovered/tasks/main.yml

- name: Patch cluster network settings
-  uri:
-    url: "http://{{ assisted_installer_host }}:{{ assisted_installer_port }}/api/assisted-install/v2/clusters/{{ ai_cluster_id }}"
-    method: PATCH
-    status_code: [201]
-    return_content: true
-    body_format: json
-    body: {
-        "cluster_networks":  [
-          {
-            "cidr": "{{ cluster_network_cidr }}",
-            "cluster_id": "{{ ai_cluster_id }}",
-            "host_prefix": "{{ cluster_network_host_prefix }}"
-          }
-        ],
-        "service_networks":  [
-          {
-            "cidr": "{{ service_network_cidr }}",
-            "cluster_id": "{{ ai_cluster_id }}",
-          }
-        ]
-    }
-
- name: Patch cluster ingress/api vip addresses
-  uri:
-    url: "http://{{ assisted_installer_host }}:{{ assisted_installer_port }}/api/assisted-install/v2/clusters/{{ ai_cluster_id }}"
-    method: PATCH
-    status_code: [201]
-    return_content: true
-    body_format: json
-    body: {
-        "cluster_network_host_prefix": "{{ cluster_network_host_prefix }}",
-        "vip_dhcp_allocation": "{{ vip_dhcp_allocation }}",
-        "ingress_vips": [{"ip": "{{ controlplane_network_ingress }}"}],
-        "api_vips": [{"ip": "{{ controlplane_network_api }}"}],
-        "network_type": "{{ networktype }}"
-    }
-


I believe this needs to be reverted as I was unable to make a cluster without including this.

Ok from my findings, we need to just retain lines 58 through 79 so we set cluster_networks and service_networks. I will investigate if we can set these on cluster creation as a follow up.

akrzos · 2025-01-17T14:51:30Z

ansible/mno-deploy.yml

+  - role: boot-iso
+    vars:
+      inventory_group: hv_vm
+      index: "{{ hybrid_worker_count }}"


I'm wondering if we could instead of including another boot-iso role here over hv_vm workers, maybe we just copy the desired number of hv_vm we want to use under workers instead. I will think of a more automated way to accomplish this as well. WDYT?

akrzos · 2025-01-17T21:15:05Z

ansible/roles/wait-hosts-discovered/tasks/main.yml

- name: Patch cluster network settings
-  uri:
-    url: "http://{{ assisted_installer_host }}:{{ assisted_installer_port }}/api/assisted-install/v2/clusters/{{ ai_cluster_id }}"
-    method: PATCH
-    status_code: [201]
-    return_content: true
-    body_format: json
-    body: {
-        "cluster_networks":  [
-          {
-            "cidr": "{{ cluster_network_cidr }}",
-            "cluster_id": "{{ ai_cluster_id }}",
-            "host_prefix": "{{ cluster_network_host_prefix }}"
-          }
-        ],
-        "service_networks":  [
-          {
-            "cidr": "{{ service_network_cidr }}",
-            "cluster_id": "{{ ai_cluster_id }}",
-          }
-        ]
-    }
-
- name: Patch cluster ingress/api vip addresses
-  uri:
-    url: "http://{{ assisted_installer_host }}:{{ assisted_installer_port }}/api/assisted-install/v2/clusters/{{ ai_cluster_id }}"
-    method: PATCH
-    status_code: [201]
-    return_content: true
-    body_format: json
-    body: {
-        "cluster_network_host_prefix": "{{ cluster_network_host_prefix }}",
-        "vip_dhcp_allocation": "{{ vip_dhcp_allocation }}",
-        "ingress_vips": [{"ip": "{{ controlplane_network_ingress }}"}],
-        "api_vips": [{"ip": "{{ controlplane_network_api }}"}],
-        "network_type": "{{ networktype }}"
-    }
-


Ok from my findings, we need to just retain lines 58 through 79 so we set cluster_networks and service_networks. I will investigate if we can set these on cluster creation as a follow up.

akrzos · 2025-01-17T21:15:49Z

ansible/roles/create-ai-cluster/tasks/main.yml

+        "additional_ntp_source": "{{ bastion_controlplane_ip if use_bastion_registry else labs[lab]['ntp_server'] }}",
+        "api_vips": [{"ip": "{{ controlplane_network_api }}"}],
+        "ingress_vips": [{"ip": "{{ controlplane_network_ingress }}"}],
+        "network_type": "{{ networktype }}"


I found the root of the original issue that was causing my deployment with this to fail PR. We can leave this as is.

akrzos reviewed Oct 4, 2024

View reviewed changes

radez force-pushed the hybrid branch 2 times, most recently from 1731073 to 2d70df6 Compare October 8, 2024 12:58

radez self-assigned this Oct 8, 2024

radez force-pushed the hybrid branch from 2d70df6 to d592245 Compare October 8, 2024 13:31

radez linked an issue Oct 8, 2024 that may be closed by this pull request

Hypervisor support for MNO clusters #536

Open

radez force-pushed the hybrid branch 2 times, most recently from faee549 to 9e0ea2c Compare October 15, 2024 19:27

radez force-pushed the hybrid branch from 9e0ea2c to 5c69752 Compare November 22, 2024 17:14

openshift-ci bot added the approved label Nov 22, 2024

radez force-pushed the hybrid branch from 5c69752 to b90e362 Compare November 25, 2024 01:10

radez force-pushed the hybrid branch 2 times, most recently from c656c10 to 22cec71 Compare December 5, 2024 17:57

akrzos reviewed Jan 2, 2025

View reviewed changes

radez force-pushed the hybrid branch from 22cec71 to 530689d Compare January 15, 2025 15:24

radez added 2 commits January 16, 2025 12:23

enable hybrid mno deployments

36d31d4

radez force-pushed the hybrid branch from 530689d to 36d31d4 Compare January 16, 2025 17:26

akrzos requested changes Jan 17, 2025

View reviewed changes

openshift-ci bot assigned akrzos Jan 17, 2025

akrzos requested changes Jan 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hybrid deployments #563

Hybrid deployments #563

radez commented Oct 3, 2024

radez commented Oct 3, 2024

akrzos Oct 4, 2024

akrzos Jan 17, 2025

radez commented Oct 8, 2024

openshift-ci bot commented Nov 22, 2024

akrzos Jan 2, 2025 •

edited

Loading

radez Jan 15, 2025

akrzos Jan 17, 2025

akrzos left a comment

akrzos Jan 17, 2025

akrzos Jan 17, 2025

akrzos Jan 17, 2025

akrzos Jan 17, 2025

akrzos Jan 17, 2025

akrzos Jan 17, 2025

akrzos Jan 17, 2025

akrzos Jan 17, 2025

akrzos Jan 17, 2025

akrzos Jan 17, 2025

akrzos Jan 17, 2025

Hybrid deployments #563

Are you sure you want to change the base?

Hybrid deployments #563

Conversation

radez commented Oct 3, 2024

radez commented Oct 3, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

radez commented Oct 8, 2024

openshift-ci bot commented Nov 22, 2024

akrzos Jan 2, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akrzos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akrzos Jan 2, 2025 •

edited

Loading